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A hypothetical model was formulated to explore factors that influenced academic and 
cliniccd achievement for graduate-entry medical students completing their third year 
of university studies. Nine latent variables were considered including the students’ 
background, previous successes with their undergraduate and postgraduate studies 
and their assessed ability to study graduate-entry medicine based on their scholastic 
aptitude and their interview selection scores. The academic and clinical achievement 
of 99 graduate-entry medical students were estimated by measuring their performance 
on two separate assessment procedures, a 150 item multiple choice examination and a 
20 item objectively structured clinical evaluation (OSCE) test. These two assessments 
were taken across two years (to include two student groups ) and were equated using 
Rasch scaling procedures. Models identifying causal pathways leading to academic 
and cliniccd achievement were tested using Partied Least Squares Path Analysis 
(PLSPAT). The study’s results suggest that medical student achievement can be 
predicted by variables, which account for 6 to 22 per cent of the variance of scores 
that assess academic achievement and cliniccd performance at the third year level 
respectively. The most significant predictors and those which had direct influence on 
graduate -entry medical student achievement were: (a) student gender, undergraduate 
grade point average scores, type of undergraduate studies undertaken, and where 
those studies were carried out that were related to the OSCE scores, and (b) whether 
or not the graduate-entry medical students had pursued other studies prior to 
undertaking the medical course and age that were both negatively related to 
achievement on the multiple choice examination. Measures of performance at 
interx’iew and student scores for GAMSAT that were used in the selection process were 
not related to the performance outcomes assessed. 

Rasch scaling, partial least squares analysis, graduate-entry, medical students, 
scholastic aptitude, scores, interview selection 


INTRODUCTION 

Three Australian university medical schools (Flinders University in South Australia, the 
University of Queensland and the University of Sydney) just over ten years ago changed their 
traditional six year conventional undergraduate medical course to a four year graduate-entry 
medical program on the grounds that they wanted to broaden the academic basis for student 
admission and acknowledge the student’s past performance in tertiary education (Aldous, Feeder, 
Price, Sefton and Tuebner, 1997). Another catalyst for the change was a widely held perception 
that secondary school results on their own did not allow university faculties to identify the 
potential students most motivated and suited to medical studies. There were demographic 
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inconsistencies also, with universities tending toward accepting metropolitan students rather than 
rurally based students into their medical programs (Field and Gordon, 2000). Other forces were 
pressing for the need to change the way Australian medical education was being managed. 
Medical science and technology were advancing at a rapid rate, giving rise to avenues of medical 
treatment that were unheard of before. Simultaneously, public expectations were also changing 
with patients wanting to be more informed and have greater engagement in their own treatment. In 
order to satisfy these demands and expectations while simultaneously wishing to prepare future 
medical practitioners more effectively, the graduate entry medical program was devised. Its 
charter was to help instil in medical students a commitment to lifelong learning, where the 
students make a greater contribution towards their own learning, become more self directed and 
employ a problem based approach to learning (Wing, 2003). 

This paper examines whether different graduate-entry medical student characteristics could be 
used to predict academic and clinical achievement in the third year (penultimate year) of medical 
studies. Graduate-entry medical students are assessed in their knowledge, their skill acquisition 
and apply their abilities to medical reasoning to clinical scenarios using two different but related 
assessment methods: completion of a 150 item multiple choice examination paper, and the 
successful execution of 20 different clinical skills scenarios, called OSCEs (Objectively 
Structured Clinical Evaluation). 

Figure 1 gives, in diagrammatic form, the path model for the latent variables shown in oval- 
shaped boxes with the directions of hypothesized causal influence being shown by the path 
arrows. Since several of the latent variables were not directly observable, the manifest variables 
used to observe the latent variables are shown in Figure 1 as rectangular boxes. Not all possible 
causal paths are shown in Figure 1, and only those hypothesized to be of sufficient magnitude to 
have recognizable influences are drawn. However, it should be noted that in testing the model, all 
possible causal pathways were examined. 

With reference to Figure 1, it is hypothesized that achievement in the student’s third year of 
medical study (latent variable 10) is directly influenced by the student’s capacity at interview 
(latent variable 9) and overall achievement in undergraduate studies (GPA) (latent variable 6) and 
involvement in post-graduate study (latent variable 7). It is also hypothesized that achievement 
could be indirectly influenced by the student’s age (latent variable 2) as students who held post- 
graduate awards would be older than students who held an undergraduate degree only. Fatent 
variable numbers 4 and 5 (place and type of undergraduate study) are thought to influence medical 
student achievement indirectly, particularly through scores obtained at interview given by the 
assessors of the interviews. In order to obtain a sufficient numbers of students for sound analyses 
this study had to employ two cohorts (students commencing in 1998 and 1999 depicted as latent 
variable 3) and it was agued that this variable would not have a direct influence on achievement 
but could influence the student’s aptitude test score (latent variable 8) and interview score (latent 
variable 9) because it was assumed that aptitude tests and interviewing procedures for course 
admission could change from year to year. 

It was argued that factors influencing student success in medicine would be best tested by 
employing a path analysis model, involving latent and manifest predictor variables. It was further 
hypothesized that any variability in performance in Year 3 of the medicine course could be 
attributed to differences in variance associated with these predictor variables. 


BACKGROUND TO THE STUDY 

While there is very little information cited in the literature about medical student characteristics 
and admission to graduate medical programs, there is a great deal of literature examining the 
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different factors that explain the undergraduate medical students’ ability to achieve both in 
multiple choice examinations and in the clinical setting. 
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Figure 1. Predicted relationships between manifest and latent variables to achievement in 
the graduate entry medical program. 


Different psychological tests have been used to predict medical student achievement. Psychiatric 
assessment of medical students has been used as a determinant of success in graduating from 
medical school but has proved to be unreliable particularly when involving such variables as the 
student’s personality, interest and attitudes (Aldrich, 1987). However, when combined with a 
profile at course entry involving student’s success in a previous degree, achievement at high 
school and grade point average such tests might be useful, as predictors of success in the non- 
clinical aspects of medical studies (Green, Peters and Webster, 1993; Hoschl and Kozeny, 1997; 
Shen and Comrey, 1997). 


The medical student’s age has been found to have effects both for gaining access to medical 
courses and for defining achievement during the medical program. Older students in several 
studies, did less well in reasoning in the sciences and tended to be graded with lower scores at the 
beginning of their courses, up to and including the first year of their medical studies (Aldous, 
Leeder, Price, Sefton and Tuebner, 1999; Huff and Fang, 1999; Kay, Pearson and Rolfe, 2002). 
Medical student ratings as given by their clinical supervisors were not significantly influenced by 
student age, yet another recent English study suggested that the older and mature aged medical 
students did achieve better overall, when compared to their younger counterparts (Rolfe, Pearson, 
Powis, and Smith, 1995; James and Chilvers, 2001). 
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Student gender seems to have some influence on medical student achievement also. Scores 
achieved by male students on medical course admission tests for GAMSAT (Graduate Australian 
Medical School Admissions Test) were higher than those achieved by female medical student 
applicants (Ramsbottom-Lucier, Johnson, and Elam, 1995; Aldous, Leeder, Price, Sefton and 
Tuebner, 1999; Chaput de Saintonge and Dunn, 2001). While on course, female medical students 
were seen generally to be at a greater risk of experiencing difficulties with their studies compared 
to their male counterparts (Huff and Fang, 1999). However, if female medical students already 
possessed an honours degree while undertaking their medical courses, problems with achievement 
were less likely (James and Chilvers, 2001). 

There has been much debate about the advantages of using grade point average scores (GPA) as a 
determinant for medical course access. GPA scores are said not to be consistent determinants of 
success in the early years of undergraduate medical studies (Blue, Gilbert, Elam and Basco, 2000). 
However, matriculation and higher undergraduate science scores were better predictors of success 
for the very beginning medical students’ achievement (Hall and Stocks, 1995; James and Chilvers, 
2001) and for predicting the student’s eventual access to a medical specialty area on graduation 
(Pringle, and Lee, 1998). Used on their own, GPA scores are not considered to be consistent 
predictors for medical course achievement especially for students who came from cultural 
minority groups (Lynch and Woode, 1990), or for students who used English as a second language 
(Chan-Ob and Boonyanaruthee, 1999), or for predicting medical students’ ability to interact with 
patients, or of estimating their performance requiring clinically related skills (Hall and Stocks, 
1995; Poussaint, 1999; Reede, 1999). 

The Graduate Australian Medical School Admissions Test (GAMSAT) modelled on the North 
American Medical College Admissions Test (MCAT) seeks to evaluate medical student abilities 
and skills developed through prior experience and learning. This includes the student’s 
understanding of basic science, general problem solving skills, critical thinking skills and writing 
ability (ACER, 1998). One recent Australian study showed that male applicants scored more 
highly on GAMSAT in reasoning in the humanities and reasoning in the sciences than female 
medical students, and that older students did less well in reasoning in the sciences. However, 
improvement in written communication occurred with age (Aldous, Leeder, Price, Sefton, and 
Teubner, 1997). 

The type of first degree studies undertaken by students has been found to have a strong influence 
on GAMSAT performance particularly for arts and social science students who did well with 
reasoning in the GAMSAT humanities test and written communication, and with science degree 
students doing better in reasoning in the GAMSAT physical sciences test. Honours students did 
better overall on GAMSAT scores than students holding just the bachelor degree (Aldous, Leeder, 
Price, Sefton, and Teubner, 1997). Although much research has been done into factors influencing 
performance in traditional medical school courses, very little research has been undertaken into 
the examination performance of students in graduate entry medical programs. 

METHODS OF INVESTIGATION AND ANALYSIS 

Participants 

A retrospective sample of 99 (consisting of two groups of 5 1 and 48 students) Australian graduate 
entry medical students who had completed their third year of medical studies was chosen for this 
research study. The two groups of students who were selected for the study, comprised those who 
commenced their four- year graduate medical program in 1998 (occasion 1) and those who started 
a year later in 1999 (occasion 2). Just over half of the medical students were males (57%) and 
with an age range from 23 years to 47 years of age (mean, 29 years and sd 6.0). 



34 Graduate-Entry Medical Student Variables that Predict Academic and Clinical Achievement 


Upon commencement of a medical course, the predominate undergraduate degree held by students 
(62%) was a science degree, followed by an applied science degree (22% of students) and with a 
Bachelor of Arts award accounting for 12 per cent of the student group. Four students held other 
qualifications such as a Bachelor of Engineering. Well over half of the group held postgraduate 
degrees (62%) in addition to their undergraduate qualifications. Most students gained their initial 
qualifications in South Australia (40%) followed by Victoria (16%), New South Wales (12%), 
with Queensland, Western Australia and Tasmania having almost equivalent but small numbers of 
students. Very small numbers of students came from the Northern Territory, the Australian 
Capital Territory and from Australian students obtaining their undergraduate degrees overseas. 

Data Collection 

With full recognition of confidentiality issues, information about student admission variables was 
obtained from past student records. Admission scores obtained for the study’s use included the 
Graduate Australian Medical School Admissions Test (GAMSAT) scores for applicants, their 
grade point average scores achieved in their undergraduate studies and the student’s interview 
scores obtained prior to course admission. 

Student achievement information in their third year of medical studies was gained from a 
retrospective audit of student records documenting results on a 150 multiple-choice examination, 
which tested different aspects of medicine in various domains including obstetrics and 
gynaecology, surgery, paediatrics and general practice. In addition the scores recorded in a 20 task 
objectively structured clinical evaluation (OSCE) were obtained. The foci of these so-called 
clinically simulated tasks included history taking, counselling skills, pharmacological 
understanding, physical examination practices and clinical dexterity in medical procedures in 
several medical domains. It should be noted that some of the multiple choice examinations and 
the objectively structured clinical evaluation tasks differed between the two occasions, but as 
some common items were used on both occasions, this allowed for the equating of the instruments 
over both time periods. 


Data Analysis 

As the student scores obtained for both the multiple choice examination and the OSCE 
assessments were estimated using classical test theory (or true-score theory), any analysis of those 
student test scores is confined to information about item difficulty and item discrimination and 
some overall estimate of the test’s reliability. This feature has serious limitations (Yen and 
Edwardson, 1999) in equating scores over time. These problems can be overcome by the use of 
item-response theory (Rasch scaling analysis) which allows an estimation of the student’s 
underlying ability to be undertaken by examining the student’s performance on a set of test items, 
after allowance has been made for the difficulty of the item and how well the items are matched to 
the student’s ability level (El-Korashy, 1995; Ludlow and Haley, 1995). The program used for this 
part of the analysis was done using the RUMM software (Andrich, Luo and Sheridan, 2001). 

As previously mentioned, the items of the multiple choice examinations and the OSCE 
assessments differed over the two years in which the study was conducted, and the sets of scores 
needed to be equated before the analysis could proceed. Data from the two different medical 
student examinations were therefore combined, as there were sets of common items linking the 
tests together over two occasions (Wright, 1993; Luo, Seow, and Chew, 2001). The analysis 
provided a calibration, standard error and fit statistic for every student and all test items involved 
in the testing. Misfitting items were eliminated in the calibrations (Andrich, Sheridan and Luo, 
1998). However, since a weak correlation between the students’ scores arising from the multiple- 
choice examination to the clinical assessments was obtained (r=+0.30). It was not possible to 
combine the two different types of scores to form a single outcome measure. 
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After test equating was completed, the Partial Least Squares Path Analysis (PLSPAT 3.01) 
program (Sellin, 1990) was used to test the separate models of variables that were hypothesized to 
influence academic achievement and clinical performance in medicine at third year level. The 
analysis would predict and identify which relationships between the variables and achievement 
might or might not exist (Noonan, and Wold, 1985). The PLSPAT procedure is highly appropriate 
for analysing and predicting relationships between educational data as it can deal with data that 
that are not normally distributed and that would be excluded in other analytical approaches. 
Additionally, PLSPAT can deal with relatively small numbers of cases and yet remain very robust 
especially in situations where not all the relevant variables are known to be normally distributed 
or where the relationship between the latent variables (theoretical constructs) and the manifest 
variables are unknown or speculative (Falk, 1987). PLSPAT is also the modelling method of 
choice as it can account for influences hypothesized to act by using causal models because it is 
clearly impossible to administer randomised controlled conditions to assess causality in such 
educational settings (Keeves, 1988). In the presentation of the findings of the analyses of the data 
collected in this study, the estimated path models are shown in diagrammatic form with only a 
brief discussion of the procedures involved in their estimation. However, only the causal paths of 
particular interest in the prediction of medical student performance are discussed although a more 
detailed report has been prepared for consideration of interested readers (Blackman, 2004). 

RESULTS 

Multiple-Choice examination 

Figure 2 shows the estimated path model for the prediction and explanation of the multiple-choice 
examination scores. 

The effects of educational status of the medical student 

The variable that describes whether or not a student entered medicine course with an award 
greater than an ordinary undergraduate degree (LV7) has a negative pathway (-0.19) on the 
achievement variable for the multiple-choice examination. This co-efficient indicates that the 
post-graduate group did significantly worse in the multiple-choice examination than students 
holding only an undergraduate award. 

The effects of the age of the medical student and success 

With respect to the relationship between the age and achievement of the multiple-choice 
examination, it can be seen that a negative path co-efficient (-0.16) exists between age and the 
achievement variable. This indicates that the younger students did significantly better on the 
multiple-choice achievement test than did older students. 

These are the only two variables that are estimated to influence directly the multiple choice 
examination scores, and a large residual path of 0.97 (representing unexplained variance) is 
recorded for the achievement variable (LV10) in Figure 2, since only six per cent of the variance 
of this outcome is explained. 


Clinical assessment scores 

Figure 3 shows the estimated path model for the prediction and explanation of the objectively 
structured clinical examination scores. 
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The effects of GP A scores 

With reference to Figure 3 it can be seen that the grade point average scores obtained by students 
in their undergraduate studies (LV6) has a direct influence on their subsequent achievement on the 
clinical assessment tasks. A positive path coefficient exists between these two variables (0.17) 
which indicates that students with higher grade point averages in their undergraduate studies 
overall achieve at a higher level on the clinical examination in their third year of medical studies. 
The loadings for the GPA scores obtained in the first (0.88) second (0.92) or third years (0.78) of 
study are an indication that there is little difference between the contributions to OSCE 
performance made by the different years of prior study. 



Figure 2. Final model for predicting achievement in the multiple-choice medical 

examination 

The effects of the type of undergraduate studies 

A path co-efficient of 0.21 exists between the clinical examination variable and the variable that 
describes the type of undergraduate study (LV5) taken by the student prior to entering medical 
studies. A positive pathway indicates that students with a science type of undergraduate degree 
(0.85) achieve better than students with an Arts degree (0.71), any other degree (0.40) or an 
applied science degree (0.0). Graduate entry medical students who held an applied science degree 
were coded as zero and employed as a dummy variable in the analysis. Students who held degrees 
in applied science did significantly worse than other students in particular clinical examination 
tasks involving history taking, interpretation of objective medical data and the specific 
management of external fixation devices to limb fracture. 

The place of undergraduate studies undertaken 

Between the achievement variable and the variable associated with where students undertook their 
undergraduate studies (LV4), a path co-efficient of 0.25 is recorded. This positive pathway 
suggests that students from certain undergraduate study sites did better than others in their 
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medical examinations. This is particularly true for students who studied for their undergraduate 
degrees in Western Australia (-0.10) and South Australia (-0.56) who ha significantly lower 
clinical examination scores compared to their counterparts who studied in Queensland (0.49). 



Figure 3. Final path model for factors influencing achievement clinical medical examination 
The effects of gender on undergraduate students 

A positive path co-efficient exists between the student gender variable (LV1) and the achievement 
variable (0.24). This indicates that female students did better than male students overall in their 
clinical assessments. A residual path of 0.88 exists after the effects of these four variables on the 
OSCE scores have been estimated, since 22 per cent of the variance of the outcome variable was 
explained. 


Discussion 


Gender and achievement 

Although there were more male students in the 1998 group than female medical students and 
more female students than male students in the 1999 group, gender was not a significant predictor 
of achievement in the multiple-choice examination, but it was for achievement in the clinical 
assessments. 

It is difficult to provide a strong explanation for the clinical achievement difference in favour of 
female students without further study. However, it might arise because the content of some of the 
OSCEs could favour female students over male students. Differential item functioning (using 
Rasch scaling) of the OSCE assessments did show that female students did significantly better 
than male students on certain clinical content areas such as infection management and paediatric 
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assessment. Alternatively, clinical ability might differ between female and male students because 
of what the OSCE examination was actually asking the medical student to do (namely, 
counselling and interpreting data) and hence female students might be more successful than the 
male students with respect to particular affective or psychomotor skills. It could also be due to the 
fact that female students interact differently with their learning environments than do male 
students and that motivation for clinical achievement differs according to whether the student is 
female or male (Chaput De-Saintonge, and Dunn, 2001). 

Student age and achievement 

A significant negative co-efficient exists between the multiple choice examination variable only, 
and the medical student’s age. This finding indicates that older students are less successful than 
younger students when taking the multiple-choice examination. This finding is consistent with 
other studies that have found that older student did less well in reasoning in the sciences (Aldous, 
Leeder, Price, Sefton, and Teubner, 1997) and were more at risk of having difficulties with 
academic work in general (Huff, and Fang, 1999). In terms of clinical performance, however, age 
was not a significant predictor with one study suggesting rather that the clinical achievement of 
older students was influenced significantly by when the clinical experience was offered. 
(Ramsbottom-Lucier, Johnson, and Elam, 1995). 

Place of undergraduate study and achievement 

The graduate-entry medical program is offered in South Australia, Sydney and Brisbane and 
students involved in this study who were from New South Wales and Queensland would have 
been able to apply to gain access to their own State’s graduate medical program but instead came 
to South Australia to study. While students from Queensland and New South Wales achieved 
higher scores in their clinical assessments, further study is required to examine how students in 
other schools of graduate-entry medicine are performing. 

Type of undergraduate award and achievement 

It was hypothesized that the type of initial degree that the student possessed would only indirectly 
influence achievement. It was thought that this variable would have its strongest influence on the 
interview and GAMS AT scores. It has been historically acknowledged that students with high 
scores in science type topics traditionally do well in medicine and this model has shown such a 
relationship exists since those students with a science degree perform better that students with an 
arts degree or another degree except an applied science degree. Students with an applied science 
degree do noticeably worse than their counterparts with other degrees. 

GPA scores and achievement 

This study has shown that GPA scores are significant predictors of the clinical focus examination. 
This pathway was predicted in the initial hypothesis and this finding concurs with other studies 
that acknowledge that undergraduate GPA scores can predict medical school performance (Blue, 
Gilbert, Elam, and Basco, 2000). 

Post-graduate award status and achievement 

Medical students who hold awards in addition to their undergraduate degree are not achieving as 
well as medical students who hold an ordinary degree in the multiple choice examination model. 
This was not predicted in the initial hypothesis, which proposed that such students would achieve 
better scores as a result of their further studies. One reason for this negative co-efficient could be 
that the multiple-choice method differed significantly from the way the students were assessed 
when they undertook their own post-graduate studies, whereas students who held an ordinary 
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degree only, were more likely to have had recent experience with written examinations, possibly 
including the multiple choice format. 

GAMS AT scores and achievement 

The GAMSAT scores in this study do not have any direct or indirect influence on student 
achievement in medical studies. The GAMSAT scores are influenced by the student age, where 
they undertook their studies and the type of undergraduate studies previously undertaken, but the 
continued GAMSAT scores as such, are not able to predict clinical or multiple choice 
examination performance. This is a consequence of the fact that the different components of 
GAMSAT do not combine effectively together and should be separated in further analyses and 
subsequent studies. 

Success at interview and achievement 

This study has been unable to identify a significant link between medical course achievement and 
scores obtained from a pre-course interview that was initially hypothesized. Admission 
performance has been linked to achievement in undergraduate medicine but only for the first year 
of medical studies according to Hall and Stocks, (1995). There is much debate about whether an 
interview should form part of the medical student admission process (Smith, 1991), whether the 
interview panels are accurate in their assessment of medical student applicants (Owen, Hayden, 
and Connors, 2002) and whether interviews should actually be linked to other admission testing 
processes (Edwards, Maldano, and Calvin, 1999). The seven domains used for student selection in 
this study are working well together as a construct to derive a combined score for course 
admission. A continuing effort to develop objective criteria for interview screening would appear 
to be required, especially if this process could further identify the medical students’ capacity for 
working in a team and detect levels of student self-confidence (Powis, Neame, Bristow, and 
Murphy, 1988; Powis, 1998). 


CONCLUSION 

Differences in the achievement of graduate-entry medical students during their third year 
assessment can be explained by different student variables, depending on the type of graduate 
entry medical student assessment used. Undergraduate grade point average scores seems to be a 
significant indicator for predicting clinical achievement, the student’s gender, the type of 
undergraduate studies undertaken and where they studied previously, and all have a direct 
influence of the clinical achievement variable. Examination scores are directly, but negatively 
influenced by the age of the student and whether or not they had undertaken further studies in 
addition to their undergraduate award. While interview and GAMSAT scores on there own, do not 
add to the explanation of student achievement in this study, they are themselves strongly 
influenced by other student variables particularly those related to the student’s previous study. 
Most predictors of performance in college and medical school leave relatively large amounts of 
variance of performance unexplained, because a highly selected sample of students is under 
investigation with greatly reduced variance. Traditional estimates using school records, GPA 
scores, information about ethnicity and gender, result in prediction equations with a maximum of 
30 per cent of variance explained, (McGanny and Ganoo, 1995). Further studies to increase the 
predictability of achievement scores could arise from the inclusion of additional variables such as 
the student’s specific ethnic group, types of clinical experience undertaken and other non- 
cognitive student factors, such as study habits, self image, degree of family support, as well as 
other non-academic activities, such as employment, hobbies and length of time in commuting to 
clinical learning areas. A mutli-level analysis which could simultaneously explore not only 
student variables but also learning institution variables (such as class sizes, timetabling, teaching 
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and learning practices) could also be employed to measure prediction and explanation of 
achievement in graduate-entry medicine. 
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